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Abstract 

Online delivery has the potential to offer significant benefits in achieving multiple goals related to sustainable 
education. For example, students from a variety of backgrounds can access educational opportunity, allowing for 
vast dissemination of education. In addition, the methods employed in online learning are generally much lower 
in carbon intensity, providing an added operational benefit to online education. Beyond these stated benefits, we 
must also identify what components of online education are deemed effective from the student’s perspective. 
This article summarizes a recent study conducted by the authors on overall student self-assessment of learning at 
a major online university, and compares these results with general pedagogical assumptions regarding the 
perceived benefits of online learning. The goal is to highlight what factors students find important in ensuring 
quality learning outcomes in the online learning environment. The extension of this work is to link successful 
components of online delivery to the general achievement of sustainability in education delivery methods. 
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1. Introduction 

The use of online learning is increasing in higher education. There are a number of reasons why this is occurring, 
including its potential to provide flexible access to content and instruction at any time, from any place and 
cost-effectiveness for institutions of higher education. Online learning can increase the availability of learning 
experiences for learners who cannot, or who choose not to, attend traditional face-to-face (onsite) offerings, it 
offers an opportunity to disseminate course content more cost-efficiently, and also enables higher student to 
faculty ratios while maintaining a level of outcome quality equivalent to face-to-face instruction. 

It has also been suggested that online learning can enhance the quality of learning experiences and outcomes by 
allowing for a complex and varied community of learners (Bransford, Brown, & Cocking 1999; Riel & Polin 
2004; Schwen & Flara 2004; Vrasidas & Glass 2004). In addition, other research has indicated the merits of 
online learning by focusing on the benefits of asynchronous discourse, suggesting such discourse is inherently 
self-reflective and therefore more conducive to deep learning than the synchronous-type discourse one would 
expect in a fully face-to-face setting (Flarlen & Doubler 2004; Hiltz & Goldman 2005; Jaffee, Moir, Swanson, & 
Wheeler 2006). 

If the preceding premises hold true, then one would expect to find student self-assessment of learning to 
generally reflect the conjectures identified above. For example, students who have engaged in “deep learning” 
would more readily identify a greater satisfaction in self-learning. These results suggest, at a minimum, that 
online learning can be an effective tool in achieving sustainability outcomes in various measures. For instance, 
based on what has been presented, online education can provide a sustainable vehicle for the delivery of 
education, while also ensuring a high degree of student satisfaction, both in terms of actual and perceived 
learning outcomes. 

To test these premises, the authors conducted a statistical analysis on a significant data set of student 
self-evaluation, attempting to discern the relationship between student self-assessment of learning and the 
teaching modality employed (onsite, online, hybrid, etc.). The results of this analysis suggest student 
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self-evaluation of learning was highest amongst onsite course offerings, followed by hybrid and then online 
offerings. While there may be many potential explanations for this trend, the trend itself suggests the conjectures 
identified above, even if true, do not necessarily correspond with student self-assessment of learning. This is an 
important trend to consider, especially where the future success of online courses is likely dependent and driven 
by student demand. While we offer potential reasons for the trends identified, further research is needed to more 
comprehensively identify the pedagogical connections between learning satisfaction and the different course 
delivery modalities current employed. 

2. Methodology 

The sample for the study consisted of 4,038 course assessment summaries over fiscal year 2009 - 2010 
year-to-date. Individual courses average 25-30 students per course. Single courses, such as independent studies, 
were excluded from the study. 

The responses taken for statistical analysis were derived from a thirty-question student assessment employed by 
the university, which contains a five point Likert scale. The thirty questions were clustered into four categories: 
student self-assessment of learning; assessment of teaching; assessment of course content; and assessment of 
web-based technology. This study focused on the results for the category of student self-assessment of learning, 
which was derived from seven of the thirty-question student assessment. An example of this category is provided 
in Table 1. For the analysis, a focus was placed on Question 4 identified in Table 1: “I gained significant 
knowledge about this subject.” The purpose was to minimize the clustering effect that potentially weighted 
certain questions to favor onsite or hybrid learning, for example, Question 2 in Table 1: “My oral communication 
skills have improved.” Oral communication skills are admittedly diminished in the traditional online learning 
environment, especially where the majority of the course activities rely on asynchronous communications. 

Course summary data of student self-assessment was sorted by academic level (undergraduate and graduate), and 
finally by course delivery modalities of Online, Flybrid, and Onsite. Each of the three major course delivery 
modalities were modified in some instances to include online supplements for onsite courses, online courses with 
synchronous audio/visual components and these represented the final fourth-level of sorting. Descriptive 
statistics were used to determine trends in both undergraduate and graduate courses across the three major course 
delivery modalities across the 2009-2010 fiscal year-to-date. Next, data was subjected to ANOVA statistical 
analysis and then, if statistically significant differences were found, to post-hoc Scheffe analysis to determine 
where those differences occurred. The data was then analyzed for trends between the different teaching 
modalities, comparing these modalities to student self-assessment of learning. 

3. Results 

The results for both undergraduate and graduate self-assessment of learning by teaching modality employed are 
shown in F igure 1. 

Figure 1 represents the variety of teaching modalities employed by National University during the 2009-2010 
academic year. The main categories of consideration are the onsite, online, and hybrid delivery methods. 
Flowever, other teaching modalities, such as the onsite with online supplement (“Onsite/Online Sup”) are 
provided to help potentially explain how various combinations of teaching modalities may combine to aid in a 
more higher student self-assessment of learning. 

The primary results indicate that both undergraduate and graduate students across various disciplines generally 
prefer onsite learning to either online or hybrid teaching modalities. Flowever, the data shows that undergraduate 
students tend to prefer hybrid to online teaching, while graduate students generally prefer online to hybrid 
teaching. In addition, there is a general trend in the data results indicating both undergraduate and graduate 
students generally score onsite forms of education delivery the highest, but also score hybrid and online 
modalities high where they are part of specialized course instruction, for example where independent studies use 
online components but still meet face-to-face, or where online courses are supplemented with synchronous 
technologies, such as ClassLive Pro (CLP). 

The differences in undergraduate student self-assessments of learning, as measured by question 4, between the 
seven different course delivery modalities is statistically significant at the .001 level. The two areas of 
statistically significant differences that are found are between “traditional” onsite v. “traditional” online 
modalities (.001 level) and between “traditional” onsite v. online courses with a synchronous communication 
component (.001 level). These online courses utilized a CLP component for supplementary instructor-student 
interactions. While not statistically significant, it is interesting to note that undergraduate students reported the 
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highest self-assessment of learning (question 4) for onsite courses that incorporate an online resource 
component. 

The differences in graduate student self-assessments of learning, as measured by question 4, between the ten 
different course delivery modalities is statistically significant at the .001 level. “Traditional'’ onsite courses are 
statistically significantly different from hybrid courses, “traditional" online courses, online courses with a 
synchronous communication component, and online practicum/project/thesis courses all at the .001 level. It is 
interesting to note that online courses taught exclusively with synchronous communication technology reported 
the highest level of student assessment of learning (question 4). 

4. Discussion 

The results obtained in the sampling data suggest student self-assessment of learning (a measure of overall 
satisfaction) is dependent on factors beyond the simple delivery of online course content. When measured 
against other modalities, online delivery of education ranks below more traditional methods of instruction, 
particularly onsite instruction. However, online courses that employ technologies that more closely mimic onsite, 
face-to-face interactions (for example, synchronous interactions via live video and audio feeds) tend to show 
higher levels of student satisfaction than entirely asynchronous online delivery. The implication for using online 
course delivery to maximize sustainable outcomes is clear; online courses should employ a mix of synchronous 
interaction opportunities to maximize student satisfaction opportunities. However, the authors recommend this 
use of increasing technology be approached with some caution. 

One of the biggest assumptions commonly made in the development of e-learning programs is that the more 
visually appealing a program, the more learning that will occur; therefore, it is easy to assume that the way to 
create a premier e-learning course is to simply add more media (such as animation, video and illustration). This 
is not necessarily true since the purpose of media elements should be to deliver the content and instructional 
methods, not to make a program merely look appealing. Visual appeal therefore is simply a byproduct of good 
instructional design. A truly premier e-learning course is one that will look attractive, feel vibrant, encourage 
participation, and incorporate activities that support the learning objectives and various learning styles of its 
participants. In addition, it will combine elements of synchronous and asynchronous learning in a way that 
maximizes student engagement while maintaining the core course objectives and goals. 

Online course delivery can be an effective way of obtaining multiple goals in sustainable education. It offers the 
benefits of educational access to a wide array of potential students, while also limiting the carbon intensity of 
course delivery. However, the benefits suggested in the literature, including the notion of “deep learning” 
through reflective asynchronous delivery methods, are not clearly reflected in this study of student 
self-assessment of learning. Rather, students seem to desire a mixed balance of synchronous and asynchronous 
delivery methods when engaging in the online environment. This is an important factor to consider when 
thinking about the overall benefits to online education in meeting sustainability goals. 

One variable that was not controlled for in this study is the impact of the particular instructor on student 
self-assessment of learning. The instructor-student relationship is an important factor in student learning, and can 
likely impact student self-assessment of learning regardless of quality of the online course design. However, we 
can assume instruction will vary equally in both online and onsite course delivery modes such that this factor 
would weigh equally in both categories of assessment. Still, further research may be appropriate to better 
understand if there are statistically significant differences between the general quality between onsite and online 
instruction. 

5. Conclusion 

Student learning is the result of the interaction of three distinct components that differ in importance in different 
course delivery modalities. The most important component across all course delivery modalities is course 
content; outstanding advanced technology and instructor competence and the ability to “connect” with students 
have little impact if course content material does not facilitate and reinforce the learning experience. The positive 
impact of efforts to constantly upgrade course content material, especially with respect to online instruction, is 
clearly seen in student assessments. Once course content has been addressed, then instructor teaching skills and 
the ability to connect with students and motivate them is likely an important component. This connecting 
between student and instruction may be seen in the data where onsite courses, or courses that emulate the onsite 
experience, reported the highest levels of student self-assessment of learning. 

The third component related to successfully student learning are the different course delivery modalities 
themselves. The data clearly show that given course content and instructor competence, modalities that afford 
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the greatest degree of instructor-student interaction provide the highest levels of student self-assessment of 
learning. Advanced technology including components such synchronous audio/video communication may 
enhance the learning experience but are likely of lesser importance than course content and instructor 
competence. 

Course assessment instillments need to be carefully reviewed and more consideration given to how component 
questions are clustered and reported. Further research needs to focus on how student and instructor interaction 
with course content elements impacts upon the student learning experience. The trend toward online v. onsite 
instruction is remarkable and has quickly impacted higher education. The online learning environment requires 
new teaching skills and an awareness of how to use emerging technology but, unfortunately, many current 
faculty members are ill prepared to successfully operate in the online environment. Schools of higher education 
must balance their efforts to develop premier e-learning courses with a corresponding reeducation program for 
current faculty to prepare them to operate in the new emerging educational paradigm. This is likely a necessary 
prerequisite to obtaining sustainable education delivery outcomes using the online environment. 
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Table 1. Example of student self-assessment of learning cluster, combining seven different assessment questions 
together to achieve a mean score of student’s self-assessment of learning. 


Ques 

Student Self-Assessment of Learning 

Score 

1 

Mv writing skills have improved 

3 S3 

*) 

My oral communication skills have improved 

3 85 

3 

My computer skills have improved 

3 85 

4 

I gained significant knowledge about this subject. 

4.85 

5 

My ability to do research has improved 

3 85 

6 

My ability to think critically about topics in this class has improved 

4 42 

7 

1 can apply what I learned in this course beyond the classroom. 

4.71 


Cluster Average: 

4.19 
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Figure 1. Trends of Self-Assessment of Learning by Teaching Modality 

Undergraduate and Graduate trends in self-assessment of learning by teaching modality for courses taught 
between July 2009 and January 2010. It is clear onsite yields the highest average self-assessment of learning in 
both the undergraduate and graduate courses examined. However, undergraduate students seem to identify 
better learning outcomes with hybrid than fully online formats, while graduate students seem to identify higher 
learning outcomes with online than hybrid teaching formats. 
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